Methods and systems for generating synthetic microstructure images of material with desired features

ABSTRACT

The disclosure generally relates to methods and systems for generating synthetic microstructure images of a material with desired features. Conventional techniques that make use of unsupervised deep generative models has no control on the generated microstructure images with specific, desired set of features. The present disclosure generates the synthetic microstructure images of the material with desired feature, by using a variational autoencoder defined with a style loss function. In the first step, the variational autoencoder is trained to learn latent representation of microstructure image of the material. In the second step, some of the dimensions of learned latent representation is interpreted as physically significant features. In the third and last step, the latent representation required for getting the desired features is appropriately sampled based on the interpretation to generate the synthetic microstructure images of the material with desired features.

PRIORITY CLAIM

This U.S. patent application claims priority under 35 U.S.C. § 119 to:Indian Patent Application No. 202121012275, filed on 22 Mar., 2021. Theentire contents of the aforementioned application are incorporatedherein by reference.

TECHNICAL FIELD

The disclosure herein generally relates to the field of materialscience, and, more particularly, to methods and systems for generatingsynthetic microstructure images of a material with desired features.

BACKGROUND

Internal structure of a material such as steel gets modified, when thematerial is put through a manufacturing process with defined processparameters, which in turn affects overall properties of the material. Alot of research have been done to investigate answers to questions suchas (i) the manufacturing process required to achieve target propertiesof the material, (ii) the target properties and the structure of thematerial that change with the process parameters, and so on. Microscopicimages associated with the microstructures of the material are typicallyused to determine complex mapping between (i) the manufacturing processwith the defined process parameters and (ii) the target properties andthe structure of the material, through the experimental characterizationHowever, cost and difficulty of the experimental characterization isoften prohibitively high. Hence, microstructure synthesis through aformal statistical characterization is considered to supportcomputational design.

Traditionally, statistical descriptors have been used to syntheticallyreconstruct the microstructures. Conventional techniques that make useof unsupervised deep generative models such as generative adversarialnets (GANs) and variational autoencoders are used as an alternative tothe traditional microstructure reconstruction methods. Theseunsupervised deep generative models try to learn compact latentrepresentations from which the original inputs may be reconstructed mostaccurately. Further, the unsupervised deep generative models are capableof synthesizing microstructures that are more realistic andstatistically equivalent to the training microstructures as compared tothe traditional methods. However, in the conventional techniques,intuitive meanings of individual dimensions of the latentrepresentations are not clear, hence there is no control on thegenerated microstructures with specific and desired set of features.

SUMMARY

Embodiments of the present disclosure present technological improvementsas solutions to one or more of the above-mentioned technical problemsrecognized by the inventors in conventional systems.

In an aspect, there is provided a processor-implemented method forgenerating synthetic microstructure images of a material with desiredfeature, the method comprising the steps of: receiving a plurality ofmicrostructure images of the material, wherein each microstructure imageof the plurality of microstructure images is of a predefined image size;dividing each microstructure image of the plurality of microstructureimages of the material, into one or more cropped microstructure images,based on the predefined image size of the corresponding microstructureimage, to obtain a plurality of cropped microstructure images from theplurality of microstructure images of the material, wherein each croppedmicrostructure image of the plurality of cropped microstructure imagesis of a predefined cropped image size; splitting the plurality ofcropped microstructure images of the material, into a plurality of traincropped microstructure images and a plurality of validation croppedmicrostructure images, based on a predefined ratio; generating a trainedvariational autoencoder, by training a variational autoencoder with theplurality of train cropped microstructure images, wherein the trainedvariational autoencoder is generated by: forming a mini batch using oneor more train cropped microstructure images out of the plurality oftrain cropped microstructure images, based on a predefined mini batchsize, to obtain one or more mini batches from the plurality of traincropped microstructure images; passing the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, to the variational autoencoder for thetraining; and performing, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder, the steps of : determining a latent vector ofeach train cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, using an encoder unitof the variational autoencoder, wherein the latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension; passing the latent vector of eachtrain cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, to a decoder unit ofthe variational autoencoder, to get a reconstructed croppedmicrostructure image of the corresponding train cropped microstructureimage; estimating a set of model weights of the variational autoencoder,by minimizing a loss function of the variational autoencoder; andupdating the variational autoencoder with the set of model weights;interpreting the trained variational autoencoder, with each croppedmicrostructure image of the plurality of cropped microstructure imagesof the material, to assign a plurality of predefined features for thecorresponding cropped microstructure image; receiving (i) a testmicrostructure image of the material, whose synthetic microstructureimages with the desired feature to be generated, and (ii) the desiredfeature, wherein the test microstructure image of the material is of apredefined test image size; and generating the synthetic microstructureimages with the desired feature, for the test microstructure image ofthe material, based on the desired feature, using the trainedvariational autoencoder.

In another aspect, there is provided a system for generating syntheticmicrostructure images of a material with desired feature, the systemcomprising: a memory storing instructions; one or more Input/Output(I/O) interfaces; and one or more hardware processors coupled to thememory via the one or more I/O interfaces, wherein the one or morehardware processors are configured by the instructions to: receive aplurality of microstructure images of the material, wherein eachmicrostructure image of the plurality of microstructure images is of apredefined image size; divide each microstructure image of the pluralityof microstructure images of the material, into one or more croppedmicrostructure images, based on the predefined image size of thecorresponding microstructure image, to obtain a plurality of croppedmicrostructure images from the plurality of microstructure images of thematerial, wherein each cropped microstructure image of the plurality ofcropped microstructure images is of a predefined cropped image size;split the plurality of cropped microstructure images of the material,into a plurality of train cropped microstructure images and a pluralityof validation cropped microstructure images, based on a predefinedratio; generate a trained variational autoencoder, by training avariational autoencoder with the plurality of train croppedmicrostructure images, wherein the trained variational autoencoder isgenerated by: forming a mini batch using one or more train croppedmicrostructure images out of the plurality of train croppedmicrostructure images, based on a predefined mini batch size, to obtainone or more mini batches from the plurality of train croppedmicrostructure images; passing the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, to the variational autoencoder for thetraining; and performing, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder, the steps of: determining a latent vector ofeach train cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, using an encoder unitof the variational autoencoder, wherein the latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension; passing the latent vector of eachtrain cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, to a decoder unit ofthe variational autoencoder, to get a reconstructed croppedmicrostructure image of the corresponding train cropped microstructureimage; estimating a set of model weights of the variational autoencoder,by minimizing a loss function of the variational autoencoder; andupdating the variational autoencoder with the set of model weights;interpret the trained variational autoencoder, with each croppedmicrostructure image of the plurality of cropped microstructure imagesof the material, to assign a plurality of predefined features for thecorresponding cropped microstructure image; receive (i) a testmicrostructure image of the material, whose synthetic microstructureimages with the desired feature to be generated, and (ii) the desiredfeature, wherein the test microstructure image of the material is of apredefined test image size; and generate the synthetic microstructureimages with the desired feature, for the test microstructure image ofthe material, based on the desired feature, using the trainedvariational autoencoder.

In yet another aspect, there is provided a computer program productcomprising a non-transitory computer readable medium having a computerreadable program embodied therein, wherein the computer readableprogram, when executed on a computing device, causes the computingdevice to: receive a plurality of microstructure images of the material,wherein each microstructure image of the plurality of microstructureimages is of a predefined image size; divide each microstructure imageof the plurality of microstructure images of the material, into one ormore cropped microstructure images, based on the predefined image sizeof the corresponding microstructure image, to obtain a plurality ofcropped microstructure images from the plurality of microstructureimages of the material, wherein each cropped microstructure image of theplurality of cropped microstructure images is of a predefined croppedimage size; split the plurality of cropped microstructure images of thematerial, into a plurality of train cropped microstructure images and aplurality of validation cropped microstructure images, based on apredefined ratio; generate a trained variational autoencoder, bytraining a variational autoencoder with the plurality of train croppedmicrostructure images, wherein the trained variational autoencoder isgenerated by: forming a mini batch using one or more train croppedmicrostructure images out of the plurality of train croppedmicrostructure images, based on a predefined mini batch size, to obtainone or more mini batches from the plurality of train croppedmicrostructure images; passing the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, to the variational autoencoder for thetraining; and performing, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder, the steps of: determining a latent vector ofeach train cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, using an encoder unitof the variational autoencoder, wherein the latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension; passing the latent vector of eachtrain cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, to a decoder unit ofthe variational autoencoder, to get a reconstructed croppedmicrostructure image of the corresponding train cropped microstructureimage; estimating a set of model weights of the variational autoencoder,by minimizing a loss function of the variational autoencoder; andupdating the variational autoencoder with the set of model weights;interpret the trained variational autoencoder, with each croppedmicrostructure image of the plurality of cropped microstructure imagesof the material, to assign a plurality of predefined features for thecorresponding cropped microstructure image; receive (i) a testmicrostructure image of the material, whose synthetic microstructureimages with the desired feature to be generated, and (ii) the desiredfeature, wherein the test microstructure image of the material is of apredefined test image size; and generate the synthetic microstructureimages with the desired feature, for the test microstructure image ofthe material, based on the desired feature, using the trainedvariational autoencoder.

In an embodiment, validating the trained variational autoencoder, usingthe plurality of validation cropped microstructure images comprises:generating a reconstructed validation microstructure image for eachvalidation cropped microstructure image of the plurality of validationcropped microstructure images, using the trained variationalautoencoder; determining a texture similarity score for each validationcropped microstructure image of the plurality of validation croppedmicrostructure images, wherein the texture similarity score for eachvalidation cropped microstructure image defines a texture similaritybetween the reconstructed validation microstructure image and thecorresponding validation cropped microstructure image; and determiningan average texture similarity score for the plurality of validationcropped microstructure images, based on the texture similarity score foreach validation cropped microstructure image of the plurality ofvalidation cropped microstructure images, to validate the trainedvariational autoencoder.

In an embodiment, interpreting the trained variational autoencoder, witheach cropped microstructure image of the plurality of croppedmicrostructure images, to assign the plurality of predefined featuresfor the corresponding cropped microstructure image, further comprising:determining the latent vector for each cropped microstructure image ofthe plurality of cropped microstructure images, using the encoder unitof the trained variational autoencoder, wherein the latent vectorcomprises the latent representation of the corresponding croppedmicrostructure image, with the predefined dimension; obtaining aplurality of variation latent vectors for each cropped microstructureimage, wherein the plurality of variation latent vectors is obtained byvarying each dimension of the corresponding latent vector in apredefined range; generating a reconstructed variation microstructureimage for each variation latent vector to obtain a plurality ofreconstructed variation microstructure images from the plurality ofvariation latent vectors, for each cropped microstructure image, usingthe decoder unit of the trained variational autoencoder; and assigningeach dimension associated with the latent vector for each croppedmicrostructure image, with a predefined feature of the plurality ofpredefined features, based on structural analysis of the correspondingreconstructed variation microstructure image.

In an embodiment, generating the synthetic microstructure images withthe desired feature, for the test microstructure image of the material,based on the desired feature, using the trained variational autoencoder,further comprising: resizing the test microstructure image of thematerial, based on the predefined test image size, to obtain a resizedtest microstructure image of the material, wherein the resized testmicrostructure image of the material is of a predefined transformedimage size; determining a test latent vector of the resized testmicrostructure image of the material, using the encoder unit of thetrained variational autoencoder, wherein the test latent vector of theresized test microstructure image of the material comprises the latentrepresentation of the resized test microstructure image, with thepredefined dimension; identifying a dimension for the resized testmicrostructure image of the material, based on the desired feature, fromclassification performed during the interpretation of the trainedvariational autoencoder; obtaining a variation test latent vector of theresized test microstructure image of the material, by varying theidentified dimension in the test latent vector of the resized testmicrostructure image of the material, in a predefined range; andgenerating the synthetic microstructure images with the desired feature,by passing the variation test latent vector of the resized testmicrostructure image of the material, through the decoder unit of thetrained variational autoencoder.

In an embodiment, the loss function of the variational autoencodercomprises a style loss function and a Kullback-Leibler (KL) lossfunction, wherein the style loss function determines a style lossbetween each train cropped microstructure image present in each minibatch of the one or more mini batches and the correspondingreconstructed cropped microstructure image, and (ii) the KL lossfunction determines a KL divergence between posterior distribution ofthe latent vector of each train cropped microstructure image present ineach mini batch of the one or more mini batches and a predefined priordistribution.

In an embodiment, the encoder unit of the variational autoencodercomprises five convolutional neural network (CNN) blocks, fourmax-pooling layers, and two parallel fully connected network (FCN)layers, wherein each CNN block is followed by a max-pooling layer exceptthe last CNN block.

In an embodiment, the decoder unit of the variational autoencodercomprises a fully connected layer, four deconvolution layers, and fourup-sampling layers, wherein each deconvolution layer is preceded by anup-sampling layer.

In an embodiment, the plurality of predefined features definesmorphological features of the material.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the embodiments of the present disclosure, asclaimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this disclosure, illustrate exemplary embodiments and, togetherwith the description, serve to explain the disclosed principles:

FIG. 1 is an exemplary block diagram of a system for generatingsynthetic microstructure images of a material with a desired feature, inaccordance with some embodiments of the present disclosure.

FIG. 2A through FIG. 2F illustrate exemplary flow diagrams of aprocessor-implemented method for generating synthetic microstructureimages of the material with the desired feature, in accordance with someembodiments of the present disclosure.

FIG. 3 is an exemplary block diagram of a variational autoencoder alongwith a loss function unit, for generating synthetic microstructureimages of the material with the desired feature, in accordance with someembodiments of the present disclosure.

FIG. 4 shows an architecture of an encoder unit of a variationalautoencoder, in accordance with some embodiments of the presentdisclosure.

FIG. 5 shows an architecture of a decoder unit of the variationalautoencoder, in accordance with some embodiments of the presentdisclosure.

FIG. 6 is a flow diagram for calculating a style loss between amicrostructure image and a reconstructed microstructure image, using aloss function unit, in accordance with some embodiments of the presentdisclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Exemplary embodiments are described with reference to the accompanyingdrawings. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears.Wherever convenient, the same reference numbers are used throughout thedrawings to refer to the same or like parts. While examples and featuresof disclosed principles are described herein, modifications,adaptations, and other implementations are possible without departingfrom the scope of the disclosed embodiments.

The present disclosure herein provides methods and systems that solvesthe technical problem of generating synthetic microstructure images of amaterial with desired feature, by using a variational autoencoderdefined with a loss function. In the first step, the variationalautoencoder is trained to learn latent representation of microstructureimage of the material. In the second step, some of the dimensions oflearned latent representation is interpreted as physically significantfeatures. In the third and last step, the latent representation requiredfor getting the desired features is appropriately sampled based on theinterpretation to generate the synthetic microstructure images of thematerial with desired features.

In the context of the present disclosure, the terms ‘microstructure’ and‘microstructure image’ may be interchangeably used, however, they referto the microscopic images of the given material.

Referring now to the drawings, and more particularly to FIG. 1 throughFIG. 6, where similar reference characters denote corresponding featuresconsistently throughout the figures, there are shown preferredembodiments and these embodiments are described in the context of thefollowing exemplary systems and/or methods.

FIG. 1 is an exemplary block diagram of a system 100 for generatingsynthetic microstructure images of a material with a desired feature, inaccordance with some embodiments of the present disclosure. In anembodiment, the system 100 includes or is otherwise in communicationwith one or more hardware processors 104, communication interfacedevice(s) or input/output (I/O) interface(s) 106, and one or more datastorage devices or memory 102 operatively coupled to the one or morehardware processors 104. The one or more hardware processors 104, thememory 102, and the I/O interface(s) 106 may be coupled to a system bus108 or a similar mechanism.

The I/O interface(s) 106 may include a variety of software and hardwareinterfaces, for example, a web interface, a graphical user interface,and the like. The I/O interface(s) 106 may include a variety of softwareand hardware interfaces, for example, interfaces for peripheraldevice(s), such as a keyboard, a mouse, an external memory, a pluralityof sensor devices, a printer and the like. Further, the I/O interface(s)106 may enable the system 100 to communicate with other devices, such asweb servers and external databases.

The I/O interface(s) 106 can facilitate multiple communications within awide variety of networks and protocol types, including wired networks,for example, local area network (LAN), cable, etc., and wirelessnetworks, such as Wireless LAN (WLAN), cellular, or satellite. For thepurpose, the I/O interface(s) 106 may include one or more ports forconnecting a number of computing systems with one another or to anotherserver computer. Further, the I/O interface(s) 106 may include one ormore ports for connecting a number of devices to one another or toanother server.

The one or more hardware processors 104 may be implemented as one ormore microprocessors, microcomputers, microcontrollers, digital signalprocessors, central processing units, state machines, logic circuitries,and/or any devices that manipulate signals based on operationalinstructions. Among other capabilities, the one or more hardwareprocessors 104 are configured to fetch and execute computer-readableinstructions stored in the memory 102. In the context of the presentdisclosure, the expressions ‘processors’ and ‘hardware processors’ maybe used interchangeably. In an embodiment, the system 100 can beimplemented in a variety of computing systems, such as laptop computers,portable computer, notebooks, hand-held devices, workstations, mainframecomputers, servers, a network cloud and the like.

The memory 102 may include any computer-readable medium known in the artincluding, for example, volatile memory, such as static random accessmemory (SRAM) and dynamic random access memory (DRAM), and/ornon-volatile memory, such as read only memory (ROM), erasableprogrammable ROM, flash memories, hard disks, optical disks, andmagnetic tapes. In an embodiment, the memory 102 includes a plurality ofmodules 102A and a repository 102B for storing data processed, received,and generated by one or more of the plurality of modules 102A. Theplurality of modules 102A may include routines, programs, objects,components, data structures, and so on, which perform particular tasksor implement particular abstract data types.

The plurality of modules 102A may include programs or computer-readableinstructions or coded instructions that supplement applications orfunctions performed by the system 100. The plurality of modules 102A mayalso be used as, signal processor(s), state machine(s), logiccircuitries, and/or any other device or component that manipulatessignals based on operational instructions. Further, the plurality ofmodules 102A can be used by hardware, by computer-readable instructionsexecuted by the one or more hardware processors 104, or by a combinationthereof. In an embodiment, the plurality of modules 102A can includevarious sub-modules (not shown in FIG. 1). Further, the memory 102 mayinclude information pertaining to input(s)/output(s) of each stepperformed by the processor(s) 104 of the system 100 and methods of thepresent disclosure.

The repository 102B may include a database or a data engine. Further,the repository 102B amongst other things, may serve as a database orincludes a plurality of databases for storing the data that isprocessed, received, or generated as a result of the execution of theplurality of modules 102A. Although the repository 102B is showninternal to the system 100, it will be noted that, in alternateembodiments, the repository 102B can also be implemented external to thesystem 100, where the repository 102B may be stored within an externaldatabase (not shown in FIG. 1) communicatively coupled to the system100. The data contained within such external database may beperiodically updated. For example, new data may be added into theexternal database and/or existing data may be modified and/or non-usefuldata may be deleted from the external database. In one example, the datamay be stored in an external system, such as a Lightweight DirectoryAccess Protocol (LDAP) directory and a Relational Database ManagementSystem (RDBMS). In another embodiment, the data stored in the repository102B may be distributed between the system 100 and the externaldatabase.

Referring to FIG. 2A through FIG. 2F, components and functionalities ofthe system 100 are described in accordance with an example embodiment ofthe present disclosure. For example, FIG. 2A through FIG. 2F illustrateexemplary flow diagrams of a processor-implemented method 200 forgenerating synthetic microstructure images of the material with thedesired feature, in accordance with some embodiments of the presentdisclosure. Although steps of the method 200 including process steps,method steps, techniques or the like may be described in a sequentialorder, such processes, methods and techniques may be configured to workin alternate orders. In other words, any sequence or order of steps thatmay be described does not necessarily indicate a requirement that thesteps be performed in that order. The steps of processes describedherein may be performed in any practical order. Further, some steps maybe performed simultaneously, or some steps may be performed alone orindependently.

At step 202 of the method 200, the one or more hardware processors 104of the system 100 are configured to receive a plurality ofmicrostructure images of the material. The material may be a processmaterial such as iron, whose synthetic microstructure images with thedesired features are required. Each microstructure image of theplurality of microstructure images is of a predefined image size. Thepredefined image size represents an image resolution (number of pixels)of the microstructure image. In an embodiment, the predefined image sizeof each microstructure image of the plurality of microstructure imagesmay be same. In another embodiment, the predefined image size of eachmicrostructure image of the plurality of microstructure images may bedifferent. Further some microstructure images of the plurality ofmicrostructure images may be of same predefined image size, while theother microstructure images may be of different predefined image size.

At step 204 of the method 200, the one or more hardware processors 104of the system 100 are configured to divide each microstructure image ofthe plurality of microstructure images of the material received at step202 of the method 200, into one or more cropped microstructure images,based on the predefined image size of the corresponding microstructureimage, such that each cropped microstructure images of the one or morecropped microstructure images is of a predefined cropped image size. Theone or more cropped microstructure images from each microstructure imageare obtained by sliding a window with a predefined stride. Thepredefined stride is defined and large enough to get reasonablydifferent patterns in each cropped microstructure image. For example,the predefined stride may be 50.

The predefined cropped image size represents the image resolution(number of pixels) of the cropped microstructure image. The predefinedcropped image size may be less than or equal to the least predefinedimage size of the microstructure image, out of the plurality ofmicrostructure images of the material received at step 202 of the method200. A plurality of cropped microstructure images is obtained from theplurality of microstructure images of the material. The predefinedcropped image size of each cropped microstructure image of the pluralityof cropped microstructure images is same. For example, the predefinedcropped image size of each cropped microstructure image may be 128×128.

At step 206 of the method 200, the one or more hardware processors 104of the system 100 are configured to split the plurality of croppedmicrostructure images of the material obtained at step 204 of themethod, into a plurality of train cropped microstructure images and aplurality of validation cropped microstructure images, based on apredefined ratio. The predefined ratio defines a number of croppedmicrostructure images out the plurality of cropped microstructure imagesto be the plurality of train cropped microstructure images and theplurality of validation cropped microstructure images. In an embodiment,the predefined ratio may be 80%:20%.

At step 208 of the method 200, the one or more hardware processors 104of the system 100 are configured to generate a trained variationalautoencoder, by training the variational autoencoder with the pluralityof train cropped microstructure images obtained at step 206 of themethod 200. FIG. 3 is an exemplary block diagram 300 of the variationalautoencoder 302 along with a loss function unit 304, for generatingsynthetic microstructure images of the material with the desiredfeature, in accordance with some embodiments of the present disclosure.

As shown in FIG. 3, the variational autoencoder 302 includes an encoderunit 302 a and a decoder unit 302 b. The loss function unit 304 includesa loss function network 304 a (not shown in FIG. 2). The encoder unit302 a of the variational autoencoder 300 takes an input microstructureimage and generates a latent vector of the input image. The decoder unit302 b of the variational autoencoder 300 takes the latent vector of theinput microstructure image and tries to generate a reconstructedmicrostructure output image for the input microstructure image, suchthat a loss calculated by the loss function unit 304, based on a lossfunction of the variational autoencoder 302, defined between the inputmicrostructure image and the reconstructed microstructure output image,is minimum.

The encoder unit 302 a of the variational autoencoder 300 includes fiveconvolutional neural network (CNN) blocks, four max-pooling layers, andtwo parallel fully connected network (FCN) layers, wherein each CNNblock is followed by a max-pooling layer except the last CNN block. FIG.4 shows an architecture of the encoder unit 302 a of the variationalautoencoder 302, in accordance with some embodiments of the presentdisclosure. As shown in FIG. 4, the encoder unit 302 a includes a firstCNN block (CB 1), a first max-pooling layer (MPL 1), a second CNN block(CB 2), a second max-pooling layer (MPL 2), a third CNN block (CB 3), athird max-pooling layer (MPL 3), a fourth CNN block (CB 4), a fourthmax-pooling layer (MPL 4), a fifth CNN block (CB 1), a first fullyconnected layer (FCL 1), and a second fully connected layer (FCL 2). Thefirst CNN block (CB 1) includes two convolution layers each with 64filters of a filter size 3×3. The second CNN block (CB 2) includes twoconvolution layers each with 128 filters of the filter size 3×3. Thethird CNN block (CB 3) includes four convolution layers each with 256filters of the filter size 3×3. The fourth CNN block (CB 4) includesfour convolution layers each with 512 filters of the filter size 3×3.The fifth CNN block (CB 5) includes four convolution layers each with512 filters of the filter size 3×3. In an embodiment, the encoder unit302 a may be obtained by removing top two fully connected layers ofVGG-19 convolutional neural network (CNN) and by adding the first fullyconnected layer (FCL 1), and the second fully connected layer (FCL 2) inparallel, on top of the VGG-19 CNN. The first fully connected layer (FCL1), and the second fully connected layer (FCL 2) are defined with a meanvector and a variance vector to learn the latent representation of theinput image.

The decoder unit 302 b of the variational autoencoder 300 includes afully connected layer, four deconvolution layers, and four up-samplinglayers, wherein each deconvolution layer is preceded by an up-samplinglayer. FIG. 5 shows an architecture of the decoder unit 302 b of thevariational autoencoder 300, in accordance with some embodiments of thepresent disclosure. As shown in FIG. 5, the decoder unit 302 b includesa fully connected layer (FCL), a reshaping layer (RSL), a firstup-sampling layer (USL 1), a first deconvolution layer (DCL 1), a secondup-sampling layer (USL 2), a second deconvolution layer (DCL 2), a thirdup-sampling layer (USL 3), a third deconvolution layer (DCL 3), a fourthup-sampling layer (USL 4), and a fourth deconvolution layer (DCL 4). Thesize of the fully connected layer (FCL) is 16348 neurons. The firstup-sampling layer (USL 1) includes 256 channels, the first deconvolutionlayer (DCL 1) includes 128 channels, the second up-sampling layer (USL2) includes 128 channels, the second deconvolution layer (DCL 2)includes 64 channels, the third up-sampling layer (USL 3) includes 64channels, the third deconvolution layer (DCL 3) includes 32 channels,the fourth up-sampling layer (USL 4) includes 32 channels, and thefourth deconvolution layer (DCL 4) includes 3 channels.

The training process of the variational autoencoder 302 with theplurality of train cropped microstructure images obtained at step 206 ofthe method 200, to generate the trained variational autoencoder isexplained in below steps. At step 208 a of the method 200, a mini batchis formed using one or more train cropped microstructure images out ofthe plurality of train cropped microstructure images, based on apredefined mini batch size, to obtain one or more mini batches from theplurality of train cropped microstructure images. The predefined minibatch size defines the number of the train cropped microstructure imagesto be present in each mini batch. If the predefined mini batch size is128, then each mini batch includes 128 train cropped microstructureimages. Each mini batch includes unique train cropped microstructureimages i.e., each train cropped microstructure image is present in onlyone mini batch.

At step 208 b of the method 200, the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, are passed to the variational autoencoder 302for the training. At step 208 c of the method 200, the steps 208 c 1through 208 c 4 are performed, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder.

At step 208 c 1 of the method 200, a latent vector of each train croppedmicrostructure image of the one or more train cropped microstructureimages present in the mini batch, is determined, using the encoder unit302 a of the variational autoencoder 302. The latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension. In an embodiment, the predefineddimension of the latent vector is 64×1. If the image size (thepredefined image size) of the train cropped microstructure image is3×128×128 (here, 128×128 represents the resolution, and 3 representscolor space), then the train cropped microstructure image processedacross the layers present in the encoder unit 302 a, to get the latentvector with the reduced predefined dimension of 64×1. More particularly,the output dimension of the first CNN block (CB 1) is 64×64×64, theoutput dimension of the second CNN block (CB 2) is 128×32×32, the outputdimension of the third CNN block (CB 3) is 256×16×16, the outputdimension of the fourth CNN block (CB 4) is 512×8×8, the outputdimension of the fifth CNN block (CB 5) is 512×4×4, and finally theoutput dimension of the latent vector is 64×1.

The encoder unit 302 a transforms feature space (spectrographicfeatures) of the train cropped microstructure image into a gaussianprobability distribution and allows random sampling from the gaussianprobability distribution to generate the latent vector in the latentspace. The first fully connected layer (FCL 1) is defined with the meanvector (μ), and the second fully connected layer (FCL 2) is defined thevariance vector (σ). In an embodiment, during at initialization stage ofthe encoder unit 302 a, the mean vector (μ) is zero and the variancevector (σ) is a unit variance vector (l). Later, the mean vector (μ) andthe variance vector (σ) are calculated based on number of input neuronsand output neurons present in each layer of the encoder unit 302 a. Theoutput dimension of the FCL 1 (the mean vector) is 64×1 and the outputdimension of the FCL 2 (the variance vector) is also 64×1. Hence theoutput dimension of the latent vector becomes 64×1.

At step 208 c 2 of the method 200, the latent vector of each traincropped microstructure image of the one or more train croppedmicrostructure images present in each mini batch of the one or more minibatches, is passed to the decoder unit 302 b of the variationalautoencoder 302, to get a reconstructed cropped microstructure image ofthe corresponding train cropped microstructure image. The decoder unit302 b transforms the latent space of the latent vector to the originalfeature space to get the reconstructed cropped microstructure image ofthe corresponding train cropped microstructure image. More specifically,the layers of the decoder unit 302 b try to re-constructs the originalfeature space having the original dimensions from the latent space. Theoriginal feature space re-constructed by the decoder unit 302 b is sameas that of the feature space of the train cropped microstructure image.

As the predefined dimension of the latent vector is 64×1, which is fedto the decoder unit 302 b, to re-constructs the original feature spaceof the train cropped microstructure image with the predefined image sizeof 3×128×128, the output dimensions of the layers present in the decoderunit 302 b are accordingly defined. The output dimension of the fullyconnected layer (FCL) is 256×8×8 (resized), the output dimension of thefirst up-sampling layer (USL 1) is 256×16×16, the output dimension ofthe first deconvolution layer (DCL 1) is 128×16×16, the output dimensionof the second up-sampling layer (USL 2) is 128×32×32, the outputdimension of the second deconvolution layer (DCL 2) is 64×32×32, theoutput dimension of the third up-sampling layer (USL 3) is 64×64×64, theoutput dimension of the third deconvolution layer (DCL 3) is 32×64×64,the output dimension of the fourth up-sampling layer (USL 4) is32×128×128, and finally the output dimension of the fourth deconvolutionlayer (DCL 4) is 3×128×128.

Conventional techniques that make use of variational auto encoderstypically use a reconstruction loss as one of the loss functioncomponent, to reconstruct the original image. However, thereconstruction loss may not be suitable to model the mismatch betweenthe microstructure image and the reconstruction. Because thereconstruction loss is equivalent to pixel-wise mean square error. Butthe pixel-wise comparison is not capable of capturing perceptual imagesimilarity between the microstructures. This is because microstructuresare of type texture images (they contain randomly repeated patterns suchas spheres, lines, and so on). The present disclosure uses the styleloss function to solve the mentioned challenges.

At step 208 c 3 of the method 200, a set of model weights of thevariational autoencoder 302 is estimated, by minimizing a loss functionof the variational autoencoder. In an embodiment, the loss function ofthe variational autoencoder includes a style loss function and aKullback-Leibler (KL) loss function. The style loss function determinesa style loss between each train cropped microstructure image present ineach mini batch of the one or more mini batches and the correspondingreconstructed cropped microstructure image. Let the train croppedmicrostructure image be x and the corresponding reconstructed croppedmicrostructure image be x′, then, the style loss function is defined inequation 1:

$\begin{matrix}{{{Style}{loss}{function}} = {{L_{style}( {{G(x)},{G( x^{\prime} )}} )} = {\sum_{l = 0}^{L}{W_{l}\lbrack {\frac{1}{4C_{l}^{2}M_{l}^{2}}{\sum_{i,j}( {G_{ij}^{i} - ( ( G_{ij}^{\prime l} )^{2} } }} \rbrack}}}} & (1)\end{matrix}$

where G(x) is gram matrices of the train cropped microstructure image x,and G(x′) is gram matrices of corresponding reconstructed croppedmicrostructure image x′.

FIG. 6 is a flow diagram for calculating a style loss between the traincropped microstructure image and the corresponding reconstructed croppedmicrostructure image, using the loss function unit, in accordance withsome embodiments of the present disclosure. As shown in FIG. 6, thetrain cropped microstructure image x and the corresponding reconstructedcropped microstructure image x′ are passed separately, to the lossfunction network 304 a present in the loss function unit 304, to obtainfeatures F^(l)(x) of the train cropped microstructure image x andfeatures F^(l)(x′) of the corresponding reconstructed croppedmicrostructure image x′, at each layer l of the loss function network304 a. In an embodiment, the loss function network 304 a may be theVGG-19 CNN

Then, an inner product is performed between the obtained featuresF^(l)(x) of the train cropped microstructure image x, at each layer l ofthe loss function network 304 a, to obtain the gram matrix G^(l)(x) ofthe train cropped microstructure image x, at the respective layer l.Similarly, the inner product is performed between the obtained featuresF^(l)(x′) of the corresponding reconstructed cropped microstructureimage x′, at each layer l of the loss function network 304 a, to obtainthe gram matrix G^(l)(x′) of the of the corresponding reconstructedcropped microstructure image x′, at the respective layer l. Then, thestyle loss between the train cropped microstructure image x and thecorresponding reconstructed cropped microstructure image x′, at thelayer l is calculated by taking L2 Norm (difference) between thecorresponding gram matrix G^(l)(x) and the corresponding gram matrixG^(l)(x′).

The KL loss function determines a KL divergence between posteriordistribution of the latent vector of each train cropped microstructureimage present in each mini batch of the one or more mini batches and apredefined prior distribution. The KL loss function is defined inequation 2:

KL loss function=D _(KL)[q _(Ø)(z|x)∥p _(θ)(z)]  (2)

where z is the latent vector of the train cropped microstructure imagex, q_(Ø)(z|x) is the posterior distribution of the latent vector z ofthe train cropped microstructure image x, corresponding reconstructedcropped microstructure image x′, p_(θ)(z) is the predefined priordistribution of the latent vector z of the train cropped microstructureimage x. The predefined prior distribution is N(0,I) where thepredefined mean vector (μ) is 0 (zero), the predefined variance vector(σ) is a unit variance vector (l), and N denotes normal distribution.

Hence, the loss function of the variational autoencoder for each pair ofthe train cropped microstructure image x and the correspondingreconstructed cropped microstructure image x′, is defined in equation 3:

Loss function=L _(style)(G(x), G(x′))+D _(KL)[q _(Ø)(z|x)∥p_(θ)(z)]  (3)

The loss function of the variational autoencoder for each pair of thetrain cropped microstructure image x and the corresponding reconstructedcropped microstructure image x′, present in one mini batch isdetermined. The set of model weights of the variational autoencoder areestimated based on the pair of the train cropped microstructure image xand the corresponding reconstructed cropped microstructure image x′,whose loss function value is minimum, for the current mini batch.

At step 208 c 4 of the method 200, the variational autoencoder areupdated with the set of model weights estimated at step 208 c 3 of themethod 200, for the current mini batch. The variational autoencoder withthe updated set of model weights is then used for the successive minibatch. Similarly, the estimated set of model weights of the variationalautoencoder are updated in each subsequent mini batch, until the one ormore mini batches are completed to get the trained variationalautoencoder. The trained variational autoencoder resulted in the step208 c 4 of the method 200 is used to generate the reconstructed imagefor any input image.

At step 210 of the method 200, the one or more hardware processors 104of the system 100 are configured to interpret the trained variationalautoencoder obtained at step 208 of the method, with each croppedmicrostructure image of the plurality of cropped microstructure imagesof the material obtained at step 204 of the method 200, to obtain aplurality of predefined features for the corresponding croppedmicrostructure image. The interpretation of the trained variationalautoencoder is performed to investigate the hidden features of thematerial present in the corresponding cropped microstructure image. Eachportion of the microstructure image may exhibit different feature of thecorresponding material. Each hidden feature of the material isinvestigated by varying the associated part present in the correspondingreconstructed image and by fixing other features.

The plurality of predefined features are morphological features that arehidden in the given material. The plurality of predefined features maybe different for each material based on the material type. For example,the material cast iron mainly contains two phases—(i) ferrite and (ii)graphite. Depending upon how the form or appearance of the graphite,different types of cast irons are defined. Within each type, differentset of features are defined. The form of the graphite also becomes afeature itself.

-   -   1. Flakes cast iron (also known as gray cast iron): The graphite        appears in the form of flakes—like black lines growing in random        directions. Following are some of the key morphological features        of the flakes cast iron:        -   Fineness of flakes: Measured by the thickness of the black            lines in microns.        -   Distribution of flakes: Interdendritic vs            non-interdendritic, i.e. do the flakes grow from a set of            common centers, or they are independent?        -   Orientation of flakes: Angle to the horizontal plane at            which the lines are oriented    -   2. Nodules or spherical cast iron: The graphite appears in the        form of spheres or nodules. Following are some of the key        morphological features of the spherical cast iron        -   Nodule size: average diameter of the nodules.        -   Shape perfection: Circularity (measured by the ratio of the            length of major axis to minor axis—a ratio of 1 means            perfectly circular).        -   Density: Number of nodules per unit area.

Hence some of the plurality of predefined features for the cast ironincludes (i) small dense spheres, (ii) small sparse spheres, (iii) largespheres, (iv) intermediate with small spheres, (v) intermediate withlarge spheres, (vi) fine flakes, (vii) sparse, medium thick flakes,(viii) thick flakes, and so on.

The interpretation of the trained variational autoencoder with eachcropped microstructure image to assign the plurality of predefinedfeatures for the corresponding cropped microstructure image is furtherexplained in below steps. At step 210 a of the method 200, the latentvector for the cropped microstructure image is determined, using theencoder unit of the trained variational autoencoder. The latent vectorcomprises the latent representation of the corresponding croppedmicrostructure image, with the predefined dimension. At step 210 b ofthe method 200, a plurality of variation latent vectors for the croppedmicrostructure image are obtained. A set of variation latent vectors isobtained by varying each dimension of the corresponding latent vector ina predefined range, to form the plurality of variation latent vectorsfrom all the dimensions of the corresponding latent vector. In this caseas the predefined dimension of the latent vector is 64×1, which mean 64dimensions, out of the 64 dimensions, each dimension is varied in thepredefined range and the remaining dimensions are fixed to obtain thecorresponding set of variation latent vectors. For example, thedimension 3 out of the 64 dimensions is varied in the predefined rangeand the remaining 63 dimensions are fixed to obtain the correspondingset of variation latent vectors, and so on. In an embodiment, thepredefined range is (−3, 3). Hence, 64 sets of variation latent vectorsare obtained to form the plurality of variation latent vectors for thecropped microstructure image.

At step 210 c of the method 200, a reconstructed variationmicrostructure image for each variation latent vector is generated, toobtain a plurality of reconstructed variation microstructure images fromthe plurality of variation latent vectors, for the croppedmicrostructure image, using the decoder unit of the trained variationalautoencoder. In this case, for the 64 sets of variation latent vectors,64 sets of reconstructed variation microstructure images are obtainedfor the cropped microstructure image.

At step 210 d of the method 200, each dimension associated with thelatent vector for the cropped microstructure image is assigned, with apredefined feature of the plurality of predefined features, based onstructural analysis of the corresponding reconstructed variationmicrostructure image. The mapping between (i) each dimension (ii) eachvariation latent vector, and (iii) each predefined feature, is performedin this interpretation step, for each cropped microstructure image ofthe plurality of cropped microstructure images. This helps inidentifying hidden features of the material during the investigation.Note here that some of the dimensions associated with the latent vectorfor the cropped microstructure image are not assigned with any featureas they may be irrelevant and does not exhibit any feature.

At step 212 of the method 200, the one or more hardware processors 104of the system 100 are configured to receive (i) a test microstructureimage of the material, whose synthetic microstructure images with thedesired feature to be generated, and (ii) the desired feature. The testmicrostructure image of the material is of a predefined test image size.The predefined test image size is the resolution of the testmicrostructure image of the material. The desired feature is one amongthe plurality of predefined features that are classified during theinterpretation of the trained variational autoencoder.

The test microstructure image of the material may already include onefeature out of the plurality of predefined features. The desired featureis one among the plurality of predefined features but is different tothe already present feature. For example, if the test microstructureimage has the feature ‘large spheres’, and the desired feature is ‘smallsparse spheres’, then the ‘large spheres’ present in the testmicrostructure image to be replaced with the ‘small sparse spheres’, togenerate the synthesized microstructure images for the testmicrostructure image.

At step 214 of the method 200, the one or more hardware processors 104of the system 100 are configured to generate the syntheticmicrostructure images with the desired feature, for the testmicrostructure image of the material received at step 212 of the method,based on the desired feature, using the trained variational autoencoderand the interpretation performed at the step 210 of the method 200. Thegeneration of the synthetic microstructure images with the desiredfeature, for the test microstructure image is further explained in belowsteps.

At step 214 a of the method 200, the test microstructure image of thematerial, is resized, based on the predefined test image size, to obtaina resized test microstructure image of the material. The resized testmicrostructure image of the material is of a predefined transformedimage size. More specifically, the predefined test image size of thetest microstructure image is resized to the predefined transformed imagesize. The predefined transformed image size is same as that of thepredefined cropped image size mentioned at step 204 of the method 200.

At step 214 b of the method 200, a test latent vector of the resizedtest microstructure image of the material, is determined using theencoder unit of the trained variational autoencoder. The test latentvector of the resized test microstructure image of the materialcomprises the latent representation of the resized test microstructureimage, with the predefined dimension. At step 214 c of the method 200,an associated dimension for the resized test microstructure image of thematerial, based on the desired feature, is identified, fromclassification performed during the interpretation of the trainedvariational autoencoder at step 212 of the method 200. Morespecifically, out of the 64 dimensions, the dimension mapped to thedesired feature during the interpretation step is identified.

At step 214 d of the method 200, a variation test latent vector of theresized test microstructure image of the material is obtained, byvarying the identified dimension at the step 214 c of the method 200, inthe test latent vector of the resized test microstructure image of thematerial, obtained at step 214 b of the method 200, in the predefinedrange used during the interpretation step. Finally, at step 214 e of themethod 200, one or more synthetic microstructure images with the desiredfeature, are generated by passing the variation test latent vector ofthe resized test microstructure image of the material obtained at step214 d of the method 200, through the decoder unit of the trainedvariational autoencoder. If the desired feature is associated with onevariation, then only one synthetic microstructure image is generated,else multiple synthetic microstructure images are generated.

Further, at step 216 of the method 200, the one or more hardwareprocessors 104 of the system 100 are configured to validate the trainedvariational autoencoder obtained at step 208 of the method 200, usingthe plurality of validation cropped microstructure images obtained atstep 206 of the method 200. The validation of the trained variationalautoencoder is performed to check an accuracy of the trained variationalautoencoder. The validation of the trained variational autoencoder withthe plurality of validation cropped microstructure images is explainedin detail in the following steps.

At step 216 a of the method 200, a reconstructed validationmicrostructure image for each validation cropped microstructure image ofthe plurality of validation cropped microstructure images, is generatedusing the trained variational autoencoder. A plurality of thereconstructed validation microstructure images is obtained from theplurality of validation cropped microstructure images. At step 216 b ofthe method 200, a texture similarity score for each validation croppedmicrostructure image of the plurality of validation croppedmicrostructure images, is determined. The texture similarity score foreach validation cropped microstructure image is determined based on atexture similarity between the reconstructed validation microstructureimage and the corresponding validation cropped microstructure image. Inan embodiment, texture similarity score is determined by using one oftexture similarity metrics (i) deep image structure and texturesimilarity (DISTS) metric, and (ii) Structural texture similarity(STSIM) metric.

The DISTS metric consists of two terms—first one compares the means offeatures maps of the reconstructed validation microstructure image andthe corresponding validation cropped microstructure image, and thesecond one computes cross covariance between the reconstructedvalidation microstructure image and the corresponding validation croppedmicrostructure image. The STSIM metric is based on a similarmodification of structural similarity metric (SSIM) to completely avoidpixel-by-pixel comparison but is computed in the Fourier spectrum.

At step 216 c of the method 200, an average texture similarity score forthe plurality of validation cropped microstructure images, is determinedby taking average of the texture similarity scores of the plurality ofvalidation cropped microstructure images obtained at step 216 b of themethod 200, to validate the trained variational autoencoder. If theaverage texture similarity score is greater than or equal to apredefined threshold, then the trained variational autoencoder obtainedat step 208 of the method 200 is efficient and accurate for the testing.If the average texture similarity score is less than the predefinedthreshold, then the trained variational autoencoder obtained at step 208of the method 200 may be retrained as mentioned at step 208 of themethod until the average texture similarity score is greater than orequal to the predefined threshold. In an embodiment, the predefinedthreshold is 0.6.

In embodiment, the validation of the trained variational autoencoder isperformed before the interpretation mentioned at step 210 of the method200 and before generating the synthetic microstructure images with thedesired feature mentioned at step 214 of the method 200. Hence thetrained variational autoencoder is efficient and accurate enough togenerate the synthetic microstructure images with the desired feature.

In accordance with the present disclosure, the methods and systems,effectively generate the synthetic microstructure images with thedesired features, with the use of the trained variational autoencoderwith the interpretation. The style loss function defined in the lossfunction enables the trained variational autoencoder to generate thesynthetic microstructure images more accurately. The generated syntheticmicrostructure images with the desired features are very helpful indetermining complex mapping between (i) the manufacturing process withthe defined process parameters and (ii) the target properties and thestructure of the material. The generated synthetic microstructure imagesare statistically equivalent to the actual microstructure images; hencethe methods and systems of the present disclosure is efficient andeffective in investigating the microstructure images for numerousapplications in the material science and engineering.

The written description describes the subject matter herein to enableany person skilled in the art to make and use the embodiments. The scopeof the subject matter embodiments is defined by the claims and mayinclude other modifications that occur to those skilled in the art. Suchother modifications are intended to be within the scope of the claims ifthey have similar elements that do not differ from the literal languageof the claims or if they include equivalent elements with insubstantialdifferences from the literal language of the claims.

It is to be understood that the scope of the protection is extended tosuch a program and in addition to a computer-readable means having amessage therein; such computer-readable storage means containprogram-code means for implementation of one or more steps of themethod, when the program runs on a server or mobile device or anysuitable programmable device. The hardware device can be any kind ofdevice which can be programmed including e.g. any kind of computer likea server or a personal computer, or the like, or any combinationthereof. The device may also include means which could be e.g. hardwaremeans like e.g. an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or a combination of hardware andsoftware means, e.g. an ASIC and an FPGA, or at least one microprocessorand at least one memory with software modules located therein. Thus, themeans can include both hardware means and software means. The methodembodiments described herein could be implemented in hardware andsoftware. The device may also include software means. Alternatively, theembodiments may be implemented on different hardware devices, e.g. usinga plurality of CPUs.

The embodiments herein can comprise hardware and software elements. Theembodiments that are implemented in software include but are not limitedto, firmware, resident software, microcode, etc. The functions performedby various modules described herein may be implemented in other modulesor combinations of other modules. For the purposes of this description,a computer-usable or computer readable medium can be any apparatus thatcan comprise, store, communicate, propagate, or transport the programfor use by or in connection with the instruction execution system,apparatus, or device.

The illustrated steps are set out to explain the exemplary embodimentsshown, and it should be anticipated that ongoing technologicaldevelopment will change the manner in which particular functions areperformed. These examples are presented herein for purposes ofillustration, and not limitation. Further, the boundaries of thefunctional building blocks have been arbitrarily defined herein for theconvenience of the description. Alternative boundaries can be defined solong as the specified functions and relationships thereof areappropriately performed. Alternatives (including equivalents,extensions, variations, deviations, etc., of those described herein)will be apparent to persons skilled in the relevant art(s) based on theteachings contained herein. Such alternatives fall within the scope andspirit of the disclosed embodiments. Also, the words “comprising,”“having,” “containing,” and “including,” and other similar forms areintended to be equivalent in meaning and be open ended in that an itemor items following any one of these words is not meant to be anexhaustive listing of such item or items, or meant to be limited to onlythe listed item or items. It must also be noted that as used herein andin the appended claims (when included in the specification), thesingular forms “a,” “an,” and “the” include plural references unless thecontext clearly dictates otherwise.

Furthermore, one or more computer-readable storage media may be utilizedin implementing embodiments consistent with the present disclosure. Acomputer-readable storage medium refers to any type of physical memoryon which information or data readable by a processor may be stored.Thus, a computer-readable storage medium may store instructions forexecution by one or more processors, including instructions for causingthe processor(s) to perform steps or stages consistent with theembodiments described herein. The term “computer-readable medium” shouldbe understood to include tangible items and exclude carrier waves andtransient signals, i.e., be non-transitory. Examples include randomaccess memory (RAM), read-only memory (ROM), volatile memory,nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, andany other known physical storage media.

It is intended that the disclosure and examples be considered asexemplary only, with a true scope and spirit of disclosed embodimentsbeing indicated by the following claims.

What is claimed is:
 1. A processor-implemented method for generatingsynthetic microstructure images of a material with a desired feature,the method comprising the steps of: receiving, via one or more hardwareprocessors, a plurality of microstructure images of the material,wherein each microstructure image of the plurality of microstructureimages is of a predefined image size; dividing, via the one or morehardware processors, each microstructure image of the plurality ofmicrostructure images of the material, into one or more croppedmicrostructure images, based on the predefined image size of thecorresponding microstructure image, to obtain a plurality of croppedmicrostructure images from the plurality of microstructure images of thematerial, wherein each cropped microstructure image of the plurality ofcropped microstructure images is of a predefined cropped image size;splitting, via the one or more hardware processors, the plurality ofcropped microstructure images of the material, into a plurality of traincropped microstructure images and a plurality of validation croppedmicrostructure images, based on a predefined ratio; generating, via theone or more hardware processors, a trained variational autoencoder, bytraining a variational autoencoder with the plurality of train croppedmicrostructure images, wherein the trained variational autoencoder isgenerated by: forming a mini batch using one or more train croppedmicrostructure images out of the plurality of train croppedmicrostructure images, based on a predefined mini batch size, to obtainone or more mini batches from the plurality of train croppedmicrostructure images; passing the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, to the variational autoencoder for thetraining; and performing, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder, the steps of: determining a latent vector ofeach train cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, using an encoder unitof the variational autoencoder, wherein the latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension; passing the latent vector of eachtrain cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, to a decoder unit ofthe variational autoencoder, to get a reconstructed croppedmicrostructure image of the corresponding train cropped microstructureimage; estimating a set of model weights of the variational autoencoder,by minimizing a loss function of the variational autoencoder; andupdating the variational autoencoder with the set of model weights;interpreting, via the one or more hardware processors, the trainedvariational autoencoder, with each cropped microstructure image of theplurality of cropped microstructure images of the material, to assign aplurality of predefined features for the corresponding croppedmicrostructure image; receiving, via the one or more hardwareprocessors, (i) a test microstructure image of the material, whosesynthetic microstructure images with the desired feature to begenerated, and (ii) the desired feature, wherein the test microstructureimage of the material is of a predefined test image size; andgenerating, via the one or more hardware processors, the syntheticmicrostructure images with the desired feature, for the testmicrostructure image of the material, based on the desired feature,using the trained variational autoencoder.
 2. The method of claim 1,further comprising validating, via the one or more hardware processors,the trained variational autoencoder, using the plurality of validationcropped microstructure images, wherein the validation comprises:generating a reconstructed validation microstructure image for eachvalidation cropped microstructure image of the plurality of validationcropped microstructure images, using the trained variationalautoencoder; determining a texture similarity score for each validationcropped microstructure image of the plurality of validation croppedmicrostructure images, wherein the texture similarity score for eachvalidation cropped microstructure image defines a texture similaritybetween the reconstructed validation microstructure image and thecorresponding validation cropped microstructure image; and determiningan average texture similarity score for the plurality of validationcropped microstructure images, based on the texture similarity score foreach validation cropped microstructure image of the plurality ofvalidation cropped microstructure images, to validate the trainedvariational autoencoder.
 3. The method of claim 1, wherein interpretingthe trained variational autoencoder, with each cropped microstructureimage of the plurality of cropped microstructure images, to assign theplurality of predefined features for the corresponding croppedmicrostructure image, further comprising: determining the latent vectorfor each cropped microstructure image of the plurality of croppedmicrostructure images, using the encoder unit of the trained variationalautoencoder, wherein the latent vector comprises the latentrepresentation of the corresponding cropped microstructure image, withthe predefined dimension; obtaining a plurality of variation latentvectors for each cropped microstructure image, wherein the plurality ofvariation latent vectors is obtained by varying each dimension of thecorresponding latent vector in a predefined range; generating areconstructed variation microstructure image for each variation latentvector to obtain a plurality of reconstructed variation microstructureimages from the plurality of variation latent vectors, for each croppedmicrostructure image, using the decoder unit of the trained variationalautoencoder; and assigning each dimension associated with the latentvector for each cropped microstructure image, with a predefined featureof the plurality of predefined features, based on structural analysis ofthe corresponding reconstructed variation microstructure image.
 4. Themethod of claim 1, wherein generating the synthetic microstructureimages with the desired feature, for the test microstructure image ofthe material, based on the desired feature, using the trainedvariational autoencoder, further comprising: resizing the testmicrostructure image of the material, based on the predefined test imagesize, to obtain a resized test microstructure image of the material,wherein the resized test microstructure image of the material is of apredefined transformed image size; determining a test latent vector ofthe resized test microstructure image of the material, using the encoderunit of the trained variational autoencoder, wherein the test latentvector of the resized test microstructure image of the materialcomprises the latent representation of the resized test microstructureimage, with the predefined dimension; identifying a dimension for theresized test microstructure image of the material, based on the desiredfeature, from classification performed during the interpretation of thetrained variational autoencoder; obtaining a variation test latentvector of the resized test microstructure image of the material, byvarying the identified dimension in the test latent vector of theresized test microstructure image of the material, in a predefinedrange; and generating the synthetic microstructure images with thedesired feature, by passing the variation test latent vector of theresized test microstructure image of the material, through the decoderunit of the trained variational autoencoder.
 5. The method of claim 1,wherein the loss function of the variational autoencoder comprises astyle loss function and a Kullback-Leibler (KL) loss function, whereinthe style loss function determines a style loss between each traincropped microstructure image present in each mini batch of the one ormore mini batches and the corresponding reconstructed croppedmicrostructure image, and (ii) the KL loss function determines a KLdivergence between posterior distribution of the latent vector of eachtrain cropped microstructure image present in each mini batch of the oneor more mini batches and a predefined prior distribution.
 6. The methodof claim 1, wherein the encoder unit of the variational autoencodercomprises five convolutional neural network (CNN) blocks, fourmax-pooling layers, and two parallel fully connected network (FCN)layers, wherein each CNN block is followed by a max-pooling layer exceptthe last CNN block.
 7. The method of claim 1, wherein the decoder unitof the variational autoencoder comprises a fully connected layer, fourdeconvolution layers, and four up-sampling layers, wherein eachdeconvolution layer is preceded by an up-sampling layer.
 8. The methodof in claim 1, wherein the plurality of predefined features definesmorphological features of the material.
 9. A system for generatingsynthetic microstructure images of a material with a desired feature,the system comprising: a memory storing instructions; one or moreInput/Output (I/O) interfaces; and one or more hardware processorscoupled to the memory via the one or more I/O interfaces, wherein theone or more hardware processors are configured by the instructions to:receive a plurality of microstructure images of the material, whereineach microstructure image of the plurality of microstructure images isof a predefined image size; divide each microstructure image of theplurality of microstructure images of the material, into one or morecropped microstructure images, based on the predefined image size of thecorresponding microstructure image, to obtain a plurality of croppedmicrostructure images from the plurality of microstructure images of thematerial, wherein each cropped microstructure image of the plurality ofcropped microstructure images is of a predefined cropped image size;split the plurality of cropped microstructure images of the material,into a plurality of train cropped microstructure images and a pluralityof validation cropped microstructure images, based on a predefinedratio; generate a trained variational autoencoder, by training avariational autoencoder with the plurality of train croppedmicrostructure images, wherein the trained variational autoencoder isgenerated by: forming a mini batch using one or more train croppedmicrostructure images out of the plurality of train croppedmicrostructure images, based on a predefined mini batch size, to obtainone or more mini batches from the plurality of train croppedmicrostructure images; passing the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, to the variational autoencoder for thetraining; and performing, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder, the steps of: determining a latent vector ofeach train cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, using an encoder unitof the variational autoencoder, wherein the latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension; passing the latent vector of eachtrain cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, to a decoder unit ofthe variational autoencoder, to get a reconstructed croppedmicrostructure image of the corresponding train cropped microstructureimage; estimating a set of model weights of the variational autoencoder,by minimizing a loss function of the variational autoencoder; andupdating the variational autoencoder with the set of model weights;interpret the trained variational autoencoder, with each croppedmicrostructure image of the plurality of cropped microstructure imagesof the material, to assign a plurality of predefined features for thecorresponding cropped microstructure image; receive (i) a testmicrostructure image of the material, whose synthetic microstructureimages with the desired feature to be generated, and (ii) the desiredfeature, wherein the test microstructure image of the material is of apredefined test image size; and generate the synthetic microstructureimages with the desired feature, for the test microstructure image ofthe material, based on the desired feature, using the trainedvariational autoencoder.
 10. The system of claim 9, wherein the one ormore hardware processors are further configured to validate the trainedvariational autoencoder, using the plurality of validation croppedmicrostructure images, by: generating a reconstructed validationmicrostructure image for each validation cropped microstructure image ofthe plurality of validation cropped microstructure images, using thetrained variational autoencoder; determining a texture similarity scorefor each validation cropped microstructure image of the plurality ofvalidation cropped microstructure images, wherein the texture similarityscore for each validation cropped microstructure image defines a texturesimilarity between the reconstructed validation microstructure image andthe corresponding validation cropped microstructure image; anddetermining an average texture similarity score for the plurality ofvalidation cropped microstructure images, based on the texturesimilarity score for each validation cropped microstructure image of theplurality of validation cropped microstructure images, to validate thetrained variational autoencoder.
 11. The system of claim 9, the one ormore hardware processors are further configured to interpret the trainedvariational autoencoder, with each cropped microstructure image of theplurality of cropped microstructure images, to assign the plurality ofpredefined features for the corresponding cropped microstructure image,by: determining the latent vector for each cropped microstructure imageof the plurality of cropped microstructure images, using the encoderunit of the trained variational autoencoder, wherein the latent vectorcomprises the latent representation of the corresponding croppedmicrostructure image, with the predefined dimension; obtaining aplurality of variation latent vectors for each cropped microstructureimage, wherein the plurality of variation latent vectors is obtained byvarying each dimension of the corresponding latent vector in apredefined range; generating a reconstructed variation microstructureimage for each variation latent vector to obtain a plurality ofreconstructed variation microstructure images from the plurality ofvariation latent vectors, for each cropped microstructure image, usingthe decoder unit of the trained variational autoencoder; and assigningeach dimension associated with the latent vector for each croppedmicrostructure image, with a predefined feature of the plurality ofpredefined features, based on structural analysis of the correspondingreconstructed variation microstructure image.
 12. The system of claim 9,wherein the one or more hardware processors are further configured togenerate the synthetic microstructure images with the desired feature,for the test microstructure image of the material, based on the desiredfeature, using the trained variational autoencoder, by: resizing thetest microstructure image of the material, based on the predefined testimage size, to obtain a resized test microstructure image of thematerial, wherein the resized test microstructure image of the materialis of a predefined transformed image size; determining a test latentvector of the resized test microstructure image of the material, usingthe encoder unit of the trained variational autoencoder, wherein thetest latent vector of the resized test microstructure image of thematerial comprises the latent representation of the resized testmicrostructure image, with the predefined dimension; identifying adimension for the resized test microstructure image of the material,based on the desired feature, from classification performed during theinterpretation of the trained variational autoencoder; obtaining avariation test latent vector of the resized test microstructure image ofthe material, by varying the identified dimension in the test latentvector of the resized test microstructure image of the material, in apredefined range; and generating the synthetic microstructure imageswith the desired feature, by passing the variation test latent vector ofthe resized test microstructure image of the material, through thedecoder unit of the trained variational autoencoder.
 13. The system ofclaim 9, wherein the loss function of the variational autoencodercomprises a style loss function and a Kullback-Leibler (KL) lossfunction, wherein the style loss function determines a style lossbetween each train cropped microstructure image present in each minibatch of the one or more mini batches and the correspondingreconstructed cropped microstructure image, and (ii) the KL lossfunction determines a KL divergence between posterior distribution ofthe latent vector of each train cropped microstructure image present ineach mini batch of the one or more mini batches and a predefined priordistribution.
 14. The system of claim 9, wherein the encoder unit of thevariational autoencoder comprises five convolutional neural network(CNN) blocks, four max-pooling layers, and two parallel fully connectednetwork (FCN) layers, wherein each CNN block is followed by amax-pooling layer except the last CNN block.
 15. The system of claim 9,wherein the decoder unit of the variational autoencoder comprises afully connected layer, four deconvolution layers, and four up-samplinglayers, wherein each deconvolution layer is preceded by an up-samplinglayer.
 16. The system of claim 9, wherein the plurality of predefinedfeatures defines morphological features of the material.
 17. A computerprogram product comprising a non-transitory computer readable mediumhaving a computer readable program embodied therein, wherein thecomputer readable program, when executed on a computing device, causesthe computing device to: receive a plurality of microstructure images ofthe material, wherein each microstructure image of the plurality ofmicrostructure images is of a predefined image size; divide eachmicrostructure image of the plurality of microstructure images of thematerial, into one or more cropped microstructure images, based on thepredefined image size of the corresponding microstructure image, toobtain a plurality of cropped microstructure images from the pluralityof microstructure images of the material, wherein each croppedmicrostructure image of the plurality of cropped microstructure imagesis of a predefined cropped image size; split the plurality of croppedmicrostructure images of the material, into a plurality of train croppedmicrostructure images and a plurality of validation croppedmicrostructure images, based on a predefined ratio; generate a trainedvariational autoencoder, by training a variational autoencoder with theplurality of train cropped microstructure images, wherein the trainedvariational autoencoder is generated by: forming a mini batch using oneor more train cropped microstructure images out of the plurality oftrain cropped microstructure images, based on a predefined mini batchsize, to obtain one or more mini batches from the plurality of traincropped microstructure images; passing the one or more train croppedmicrostructure images present in each mini batch, at a time, out of theone or more mini batches, to the variational autoencoder for thetraining; and performing, for the one or more train croppedmicrostructure images present in each mini batch, at a time, until theone or more mini batches are completed, to generate the trainedvariational autoencoder, the steps of: determining a latent vector ofeach train cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, using an encoder unitof the variational autoencoder, wherein the latent vector comprises alatent representation of the corresponding train cropped microstructureimage, with a predefined dimension; passing the latent vector of eachtrain cropped microstructure image of the one or more train croppedmicrostructure images present in the mini batch, to a decoder unit ofthe variational autoencoder, to get a reconstructed croppedmicrostructure image of the corresponding train cropped microstructureimage; estimating a set of model weights of the variational autoencoder,by minimizing a loss function of the variational autoencoder; andupdating the variational autoencoder with the set of model weights;interpret the trained variational autoencoder, with each croppedmicrostructure image of the plurality of cropped microstructure imagesof the material, to assign a plurality of predefined features for thecorresponding cropped microstructure image; receive (i) a testmicrostructure image of the material, whose synthetic microstructureimages with the desired feature to be generated, and (ii) the desiredfeature, wherein the test microstructure image of the material is of apredefined test image size; and generate the synthetic microstructureimages with the desired feature, for the test microstructure image ofthe material, based on the desired feature, using the trainedvariational autoencoder.